Obsidian Vault

02 - DataEngineer

Spark - Data preprocessing1

CanVas

SparkSessionCollection

TransformAction

DrawChat

TransformAction

images

Pasted image 20240613140948

Pasted image 20240613151356

Pasted image 20240613152541

Pasted image 20240613160921

Pasted image 20240613162030

Pasted image 20240613162216

Pasted image 20240613174955

Pasted image 20240615153209

Pasted image 20240615153937

Pasted image 20240618070836

Pasted image 20240618071010

Pasted image 20240618072852

Pasted image 20240618144915

Pasted Image 20240618151714_691

Pasted image 20240618152039

Pasted image 20240618152453

Pasted image 20240618154633

Pasted image 20240618155701

Pasted image 20240618155733

Pasted image 20240618170427

Pasted image 20240618172352

Pasted image 20240618190042

Spark-StructureAPI_Framework

SPark DataSource and Sinks

Spark RDDS API's

Spark-StructureAPI_Framework

Spark - Data preprocessing1

Spark Transformation Actions

Spark_ExecutionPlan

Spark_log4jconfig

Spark_Session_Configuration

Spark RDDS API's

Pasted image 20240619165833.png

if 1 of the RDD partition failed or crashes.

Pasted image 20240619174326.png
Pasted image 20240619180129.png

This tells about what is RDD

How to Create RDD

![[Pasted image 20240620113012.png]] ![[Pasted image 20240620113110.png|1000]]

How to Process RDD ?

![[Pasted image 20240625204150.png]] Output is list

Table Of Contents

Spark RDDS API's

How to Create RDD

How to Process RDD ?